Deriving Value from Patient Flow Analytics

Session 200, February 14, 2019

Justin Boyle & Sankalp Khanna

CSIRO Australian e-Health Research Centre

www.csiro.us

Justin Boyle, BEng(Hons1) PhD, Principal Research Scientist

Has no real or apparent conflicts of interest to report.

Sankalp Khanna, BEng,MInfTech, PhD, Senior Research Scientist

Has no real or apparent conflicts of interest to report.

Conflict of Interest

• Learning Objectives

• Introduction – Where we are from

• The Australian Health System – Why we do what we do

• Patient Flow Analytics @ CSIRO – Our approach

• Case Studies

Agenda

• Identify strategies to ensure effective impact from developed

health analytics solutions

• Form effective partnerships between problem solvers and problem

owners to ensure success

• Apply appropriate statistical rigor in planning and executing data

science projects

• Identify strategies to decongest the health system

Learning Objectives

Australia’s National Science Agency

Over 5000 Research Scientists

58 Sites globally, Research activities in 80 Countries

$1 Billion Annual budget

Top 1% Of Global research agencies

Hosts Boeing’s largest R&D facility outside of the US

Run NASA’s spacecraft tracking facilities in Australia

Invented WiFi, used in five billion devices globally.

CSIRO: Commonwealth Scientific

and Industrial Research Organisation

CSIRO’s Digital Health Research Program

BIOMEDICAL

INFORMATICS

Biostatistics, imaging and

genomics based -clinical

workflows

How: Leveraging operational &

clinical data through analytics,

modelling, decision support &

automation

HEALTH

INFORMATICS

Improving health system

performance & productivity from

electronic health data

How: Meaningful data

interoperability and analysis for

decision support, analytics,

modelling and reporting

HEALTH

SERVICES

Improving access to services &

management of chronic

diseases

How: Service delivery models

utilising telehealth, mobile

health & remote monitoring

The Australian Health System

Over the Past DecadeOver the Past Decade

Health

Spending

Population

Growth

50%50% 17%17%

Reference: Australia's health 2018, https://www.aihw.gov.au/reports/australias-health/australias-health-2018/

Overcrowding in hospitals: an

international crisis

- Increased wait times. - Increased walkouts.

- Increased medical errors. - Ambulance diversion.

- Increased length of stay. - Patient safety at risk.

- Increased medical negligence claims - Unnecessary deaths.

admissions

Elective

surgery

Reference: American College of Emergency Physicians, Emergency Department Crowding: High-Impact Solutions, 2008

Do we have a problem with Patient

Flow ?

OVERCROWDED HOSPITALS

AGEING POPULATIONS

EXPLODING HEALTH BUDGETS

AMBULANCE RAMPING

LONG WAITING TIMES

STAFFING SHORTAGES

Image Source: Fairfax Media

Fixing Patient Flow - Why does it matter?

Reference: Boyle J, Zeitz K, Hoffman R, Khanna S, Beltrame J. Probability of severe adverse events as a function of hospital occupancy. IEEE J Biomed Health Inform. 2014 Jan;18(1):15-20

0.7 0.75 0.8 0.85 0.9 0.95 1 1.05 1.1 1.15

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Occupancy

Probability of SAC1/SAC2 Incident

No event/day

1 event/day

2 events/day

3 events/day

Patient Flow Analytics @ CSIRO

Improving public hospital performance

through efficiency improvements

Improving public hospital performance

through efficiency improvements

Creating an evidence base to support

policy and decision making

Creating an evidence base to support

policy and decision making

• Better Capacity Management

• 85% occupancy delivers optimum patient flow

• Higher levels result in increased patient risk and regular overcrowding

• Is 85% a one size fits all ?

• How do I manage my hospital capacity ?

• How unsafe is my crowded hospital ?

• Early Discharge

• Early discharge should help ease crowding.

• Where is the evidence ?

• How much benefit could it potentially deliver ?

Hospital Crowding: The Magic Fix

Q > How full can I run my Hospital

at before Capacity Crisis ?

Case Study 1 : Capacity Management

ED, Admission & Discharge vs occupancy

3 key choke points - performance declines:

• A - Admission/discharge surge

• B - ED overwhelmed

• C - Admissions overwhelmed

Overcrowding affects :

 Access Block

 ED Length of Stay (Inpatients)

 Inpatient Length of Stay

 Inpatient Admissions from ED

 ED Length of Stay (not admitted)

Q > Do we see this trend across hospitals of all sizes ?

Case Study 1 : Capacity Management

71%

80%

89%

98%

107%

116%

OCCUPANCY

Inpatient Admissions (patients/hr) (Y1 axis)

Inpatient Discharges (patients/hr) (Y1 axis)

ED Presentations (patients/hr) (Y1 axis)

ED Discharges (patients/hr) (Y1 axis)

Inpatient Admissions from ED (patients/hr) (Y1 axis)

Inpatient Length of Stay (days) (Y2 axis)

ED Length of Stay (inpatients) (hours) (Y2 axis)

ED Length of Stay (others) (hours) (Y2 axis)

ED Access Block Cases (inpatients) (patients/hr) (Y2 axis)

75%

80%

85%

90%

95%

100%

105%

110%

115%

OCCUPANCY

GROUP 3

300 >= Beds

75%

80%

85%

90%

95%

100%

105%

110%

OCCUPANCY

GROUP 2

900 >= Beds > 300

70%

75%

80%

85%

90%

95%

100%

OCCUPANCY

GROUP 1

Beds > 900

YES … but at different levels

Group 1 (Large hospitals) :

• Choke Point A : 86%

• Choke Point B : 90%

• Choke Point C : 94%

Group 2 (Mid-size hospitals) :

• Choke Point A : 90%

• Choke Point B : 96%

• Choke Point C : 101%

Group 3 (Small hospitals) :

• Choke Point A : 98%

• Choke Point B : 102%

• Choke Point C : 106%

Case Study 2 : Discharge Timing

Q > Does discharge peak timing affect ED LOS and Access Block

5 hours

Admissions

Discharges

‘d1’

5 hours

Discharges

‘d2’

Category 1 Category 2 Category 4 Category 5

Category 3

Hour of Day

Number of Patients

Define

discharge

peak timing

Reference: Khanna S, Boyle J, Good N, Lind J, Impact of Admission and Discharge Peak Times on Hospital Overcrowding, Proc. 19

Australian National Health Informatics Conference (HIC 2011), 2011, 82-88

Case Study 2 : Discharge Timing

Q > Does discharge peak timing affect ED LOS and Access Block

Reference: Khanna S, Boyle J, Good N, Lind J, Impact of Admission and Discharge Peak Times on Hospital Overcrowding, Proc. 19

Australian National Health Informatics Conference (HIC 2011), 2011, 82-88

100

150

200

250

75%

80%

85%

90%

95%

100%

105%

110%

115%

Access Block Cases per day

Occupancy (%)

Category

23 Hospitals

Mean LOS (days)

Mean EDLOS (hours)(Y2)

Cat 5 vs Cat 1

• 13% Higher Peak

Occupancy

• 60 cases/day higher

Access Block

• 0.7 hours higher

Mean ED LOS

Case Study 2 : Discharge Timing

100

150

200

250

300

350

400

450

100

105

Discharges/hour

Occupancy (%)

Time of Day (hour)

2 Hours Early

1 Hour Early

Actual

1 Hour Late

2 Hours Late

2 Hour Early Discharge (all 23 Hospitals) :

• Average Occupancy reduced from 93.7% to

91.6%.

• Maximum Occupancy reduced from 110.8%

to 106.1%.

• Time spent above 95% occupancy reduced

from 34.7% to 21.5%.

Q > Can we quantify the impact of Early Discharge ?

Q > What happens if overcrowding delays Discharge ?

Reference: Khanna et al. Unravelling relationships: Hospital occupancy levels, discharge timing and emergency department access block. Emerg Med Australas. 2012 Oct;24(5):510-7

Case Study 2 : Discharge Timing

Q > How does this affect my KPIs ?

Q > How can I operationalise this ?

Scenario

Description

50% of patients to be discharged by 10am, 80% by 12pm and 100% by

2pm.

35% of patients to be discharged by 11am, 70% by 2pm and 100% by 5pm.

50% of patients to be discharged by 11am, 70% by 2pm and 100% by 5pm.

80% of patients to be discharged by 11am.

40% of patients to be discharged by 10am, 70% by 2pm, 90% by 5pm and

100% by 10pm.

Select the same patients as for

Scenario 5 but change only the emergency

discharge times, leaving elective patient discharge times unchanged.

Select the same patients as for Scenario 3 but change only the emergency

discharge times, leaving elective patient discharge times unchanged.

Reference: Khanna S, Sier D, Boyle J, Zeitz K. Discharge timeliness and its impact on hospital crowding and emergency department flow performance. Emerg Med Australas. 2016 Apr;28(2):164-70

Case Study 2 : Discharge Timing

Q > How does this affect my KPIs ?

Q > How can I operationalise this ?

Reference: Khanna S, Sier D, Boyle J, Zeitz K. Discharge timeliness and its impact on hospital crowding and emergency department flow performance. Emerg Med Australas. 2016 Apr;28(2):164-70

% change as compared to baseline

Scenario

Scenario Description

NEAT

Performance

Ave Bed

Occupancy

Ave Inpatient

LOS

Ave wait for

Inpatient Bed - ED

Ave wait for

Inpatient Bed - All

50%

by 10am, 80% by

12pm,

100

% by 2pm.

+16.1% -1.5% -1.7% -25.5% -24.2%

35%

by 11am, 70% by 2pm,

100

% by 5pm.

+5.7% -0.2% -0.3% -6% -5.7%

% by 11am, 70% by 2pm ,

100

% by 5pm.

+9.4% -0.5% -0.5% -11.8% -10.5%

80%

by 11am. +16.2% -1.5% -1.6% -24.9% -23.5%

40%

by 10am, 70% by 2pm,

90% by

5pm, 100% by 10pm.

+7.3% -0.3% -0.4% -8.6% -7.7%

Same as Scenario 5

but ED only

+6.9% -0.2% -0.3% -7.3% -6.4%

Same as Scenario 1

but ED only

+15.7% -1.2% -1.3% -22.7% -20.5%

How Many of What Beds?

Access Target:

ED LOS<4hrs

Case Study 3 : Predicting Demand

• Forms a regular component

of daily bed management

across major QLD public

hospitals

• Licensed in Australia and

overseas

• Estimated to deliver $23

million (direct), and $250m

(indirect) in productivity gains

per annum

• Several awards related to

efficiency and effectiveness

References:

• Boyle J, Jessup M, Crilly J, et al. Predicting emergency department admissions.

Emerg Med J. 2012 May;29(5):358-65.

• Boyle J, Ireland D, Webster F, O’Sullivan K, Predicting Demand for Hospital Capacity Planning,

Conf Proc IEEE Biomedical and Health Informatics. 2016: 328-331

• Jessup M, Crilly J, Boyle J, et al. Users' experiences of an emergency department patient

admission predictive tool: A qualitative evaluation. Health Informatics J. 2016 Sep;22(3):618-32

Case Study 3 : Predicting Demand

• Forms a regular component

of daily bed management

across major QLD public

hospitals

• Licensed in Australia and

overseas

• Estimated to deliver $23

million (direct), and $250m

(indirect) in productivity gains

per annum

• Several awards related to

efficiency and effectiveness

References:

• Boyle J, Jessup M, Crilly J, et al. Predicting emergency department admissions.

Emerg Med J. 2012 May;29(5):358-65.

• Boyle J, Ireland D, Webster F, O’Sullivan K, Predicting Demand for Hospital Capacity Planning,

Conf Proc IEEE Biomedical and Health Informatics. 2016: 328-331

• Jessup M, Crilly J, Boyle J, et al. Users' experiences of an emergency department patient

admission predictive tool: A qualitative evaluation. Health Informatics J. 2016 Sep;22(3):618-32

Case Study 3 : Making it Work

WHO?

• Bed manager, after hours co-

ordinator, hospital executive,

hospital executive on-call, Director

ED, Director Medicine, Director

Surgery, decision support services

WHY?

• Inform managers (nursing and

medical) re expected admissions

and discharges so they have

information to work proactively

HOW?

• PAPT Software and PAPT

Procedure Manual: guide to assist

communication processes with in-

built ‘triggers’ to inform decision

making (planning and functioning)

WHEN?

• Daily and Weekly

Reference: Crilly J, Boyle J, Jessup M, et al. The Implementation and Evaluation of the Patient Admission Prediction Tool: Assessing Its Impact on

Decision-Making Strategies and Patient Flow Outcomes in 2 Australian Hospitals. Qual Manag Health Care. 2015 Oct-Dec;24(4):169-76

Case Study 3 : Making it Work

WHO?

• Bed manager, after hours co-

ordinator, hospital executive,

hospital executive on-call, Director

ED, Director Medicine, Director

Surgery, decision support services

WHY?

• Inform managers (nursing and

medical) re expected admissions

and discharges so they have

information to work proactively

HOW?

• PAPT Software and PAPT

Procedure Manual: guide to assist

communication processes with in-

built ‘triggers’ to inform decision

making (planning and functioning)

WHEN?

• Daily and Weekly

Reference: Crilly J, Boyle J, Jessup M, et al. The Implementation and Evaluation of the Patient Admission Prediction Tool: Assessing Its Impact on

Decision-Making Strategies and Patient Flow Outcomes in 2 Australian Hospitals. Qual Manag Health Care. 2015 Oct-Dec;24(4):169-76

Case Study 4 : Evidence Driven KPIs

The National Emergency Access Target (NEAT)

“By 31 Dec 2015, 90% of all patients will physically leave the

Emergency Department (ED) within 4 hours”

Australian Institute of Health and Welfare Canberra, Australian

hospital statistics 2012–13, Emergency department care,

Health Services Series Number 52

64%

66%

72%

77%

66%

67%

64%

57%

64%

66%

72%

77%

66%

67%

64%

57%

• First study to deliver evidence driven

targets for public hospital ED patient flow

• Directly translated into government policy

• Several awards and endorsements

• Replicated in several other states, and in

the UK

Reference: Sullivan C, Staib A, Khanna S, et al. The National Emergency Access Target (NEAT)

and the 4-hour rule: time to review the target, Medical Journal of Australia 2016 May 16;204(9):354

Case Study 4 : Evidence Driven KPIs

Methodology :

• Focus on Emergency Admissions only

(i.e. eHSMR)

• Exclude Palliative Care and Short Stay

• Develop several predictive models for

eHSMR calculation

• Model relationship between NEAT

Compliance and eHSMR

• Check for confounding effect of palliative

care and short stay

No robust evidence regarding a clinically

significant mortality benefit above this threshold

Reference: Sullivan C, Staib A, Khanna S, et al. The National Emergency Access Target (NEAT)

and the 4-hour rule: time to review the target, Medical Journal of Australia 2016 May 16;204(9):354

Case Study 5 : Reducing Readmissions

Project Brief

• To develop, implement and evaluate a

web-based risk stratification algorithm

that can be used in-hospital to identify

chronic disease patients with a high risk

of re-hospitalisation.

What are we predicting

• Unplanned re-admission within 30

days of discharge from hospital

• Unplanned ED re-presentation within

30 days of discharge from hospital

Timeline

• Trial - Apr 2018 to Mar 2019

• Evaluation - Apr 2019 to Jun 2019

Chronic Disease Patient Admitted

to Hospital

Risk Score generated overnight

Risk score used by care teams for

appropriate interventions and

care/discharge planning

Next morning

Case Study 5 : Reducing Readmissions

Patients who, over a 5 year period, as an

Emergency or Admitted Patient :

• Attended Logan Hospital, and

• Had at least one Chronic Disease visit

(any QLD hospital)

Patient Cohort

• Emergency Data (EDIS/FirstNet)

• Inpatient Data (QHAPDC/ePADT)

• Mortality Data (Death Registry)

• Pharmacy dispensing information (eLMS)

• Pathology test results (AUSLAB)

Data Used for Modelling & Validation

• Patients stays in prev. 180 days

• ED presentations in prev. 180 days

• Marital status

• Age

• Indigenous status

• SEIFA

• Admission source

• Admission unit

• Care type

• Elective status

• Planned same day status

• Binary flags for routine dialysis

• Number of medication records

• Binary flags for medication

• Binary flags for abnormal pathology

Predictor Variables in Final Models

• Logistic regression

• Naïve Bayes

• Neural Nets

• Random Forests

• Generalised Boosting

Modelling Techniques

Case Study 5 : Making it Work

Employing Intelligible Machine Learning

Case Study 5 : Making it Work

Employing Intelligible Machine Learning

- Engagement between with clinicians when designing solutions

- Focus on translation early on

- Frameworks, Standards and Governance

- Ensuring statistical rigor

- Choosing the right outcome measures – KPIs vs patient outcomes

- Translation is not the clients problem

- Innovative approaches

- Value of understanding the domain and the data

- Empathy with perspectives of the problem owner and the pain points

Recommendations

For more information, please contact:

www.csiro.us

Please remember to complete the online session evaluation

Questions

Justin Boyle

Principal Research Scientist

e Justin.Boyle@csiro.au

Sankalp Khanna

Senior Research Scientist

e Sankalp.Khanna@csiro.au

t @SankalpKhanna

l /in/sankalpk